Stepper Symphony

Introduction

Our goal in this project was to create music using stepper motors. By controlling the speed and duration of a stepper motor’s rotation, we can control the pitch that it makes and play musical notes. This is the basis for our project. Our goal is to be able to play different notes on the stepper motors through using a keyboard, simulating a piano.

Design and Testing

In order to complete this project, we split it up into the following steps, so that we could have concrete goals set out for each lab session: selecting motors, tuning them to different notes, setting up keyboard inputs, and connecting keyboard inputs to musical notes.

The first step was to determine which motors we would be using, which we did during our first work session. Our first thought was to use the DC motors from Lab 3, but these lacked the precision we needed. We also had available to us a small, cheaper stepper motor, but we found that these weren’t loud enough to be heard clearly. Our last option was to buy more powerful NEMA stepper motors, which we would then test during our next work session. To test out the motors, we hooked them up to power and wrote up a simple python script to control them. However, the different motors had different motor drivers and thus different code, which meant that not much of this test code could be reused later on. We initially had issues with getting the motors to run correctly, which we debugged using a multimeter to check connections and voltages, and the current reading on the power supply to determine if our system was being powered.

Once we had our hardware, we set it up as shown in Fig. 1 and 2 but for only one motor, referencing the data sheet [1] for the motor and motor driver connections. The RPi controls the motors using two GPIO pins for each: STEP which turns the motor, and DIR which controls its direction. The connection between the RESET and SLEEP pins is also of particular importance as this serves as an “enable”, and is required for the motors to run. Something else we realized was that without solid electrical connections, the motors do not receive enough power and don’t turn correctly, resulting in weird noises. As a result, the jumper wire and breadboard circuits we had previously been using were not adequate, and we recreated our system on a soldered protoboard.

Fig. 1: Motor Driver’s connection to the Raspberry Pi and to the Power Supply. [1]

Fig. 2: Overall Hardware Diagram

With our hardware setup secure, we could start tuning our motors, which we did using a python script (nema_control.py) we created based off of some sample code [6]. The basis of this script is the function note(), which plays a note for a specified duration. note() calculates a pause duration Dell based on a value (correlated to period) associated with the desired note, as well as a loop duration Coun depending on how long the note should be played for. Then, note()drives the STEP GPIO pin high and low for the loop duration, placing a pause of Dell microseconds between each GPIO change. To tune, we set variables for all notes in the fourth octave in our code, and tested each note by playing it through the stepper motor. While we could have used frequency ratios to calculate the values for each note, this ended up being out of tune, so we manually tuned these values using this python script and an instrument tuner app [5]. For additional safety, we programmed one of the buttons (GPIO 27) on the piTFT to be a quit button, which would allow us to quit if our code gets stuck, avoiding a forced reboot.

Fig. 3: Mapping Motor Steps to Frequencies and Musical Notes [5]

Next, we worked on creating a user interface for our system. We decided on using a computer keyboard as this not only is a simple way to connect many inputs to the RPi, but it also would be similar to playing notes on a piano. The keyboard connects to the RPi as shown in Fig. 2, and in order to detect notes we used the Pygame module, specifically Pygame.event and Pygame.key [4]. Pygame.key.get_pressed() returns an array of the states of all keys on the keyboard, with a one representing a key that is pressed and a zero for one that isn’t. Using a script called keyboard_control.py, we can determine which keys are pressed by getting the nonzero indices of get_pressed(), which is updated every time any input is detected from the keyboard (key down/up). Not only does this script serve as an initial functionality test, but it also allows us to get the indices of the keys we want to use in the get_pressed() array. We mapped keys to notes as shown in Fig. 5 using a python dict, with the keys being the get_pressed() indices and the values being the associated note. As an initial integration test (piano_controlv1.py), we set each note to play for a hard coded duration after each keypress, though for our final design we want the note to play while the key is held down. We also set up the hardware for the two remaining stepper motors.

Lastly, we worked properly integrating the keyboard, programming it such that a note is played for as long as a given key is pressed. This is done in 3piano_control.py. We started by programming this using threads and a state variable for the motor: its value would either correspond to a note that the motor will play, or be 0, indicating that the motor should be silent. The thread would run the motor control loop while the main loop would be used to detect keyboard controls. However, when we implemented this the motor sounded way off. As it turns out, because both the thread and the main loop were running on the same core, the maximum frequency at which we could drive the motor was severely limited. This problem would only get worse as we added two more threads for the remaining motors so we had to find a different solution: Python’s multiprocessing library [2]. We programmed the RPi such that the first core was used to detect keyboard inputs and handle their logic. The second core then controlled the first motor, the third core controlled the second motor, and, lastly, the fourth core controlled the third motor. The first core can be considered the “main” core, as it runs our main while() loop, while the other cores run defined processes. This fixed our issue, and the stepper motors were able to play the notes that we had tuned.

To integrate all three motors, we had to figure out how to allocate inputs to available motors. Our logic to control the motors was as follows: the first motor would be the primary motor, playing the first detected note if it is not already being used; if a user was only playing one note at a time, these notes would be played on this motor. If the user started playing a second note, the code would detect that the first motor is already occupied and the second motor would start playing. Similarly, if the first and second motors were busy and the user played a third note, the third motor would be used. If the user played four or more notes at a time, the code would detect that all motors are busy and not react to subsequent notes. This logic updates whenever any input is detected from the keyboard (eg. key up or key down), so the user can quickly switch between notes, even if another note is being held down.

Fig. 4: Final Code Design

Fig. 5: Keyboard Control Keys

After we had all three motors working and the functionalities figured out successfully, we added more buttons for fun! We configured a few keys on the keyboard to allow the user to move up or down an octave by using a multiplicity factor. We also programmed the “K”, “O”, and “L” keys to be the C, C#, and D notes of the higher octave for easier access, which is useful when playing certain songs. We also added the title of our project on our previously-blank PyGame window.

Results

When we first started to work on this project, we had two main goals: first, to be able to play music using a keyboard; and second, to be able to play midi files through stepper motors. Over the course of the last few weeks, we were able to successfully complete our first goal, as described in the section above. We, however, did not have enough time for our second goal so we focused on polishing up the first. When we were initially creating our proposal, we were aware that we may not have enough time to complete both goals, so the second goal is something we would like to work on in the future. If we had more time, we would also have worked on the PyGame screen to add instructions on how to use our project and set up crontab so that we could schedule our program to run on startup. Overall, our project was successful and we were able to solve any hiccups we encountered.

Fig. 6: Hardware Photos - Overall System

Fig. 7: Hardware Photos - Motor Drivers

Conclusions

We learned the importance of timing through this project. In a system like ours in which we aimed to play an instrument, we had to ensure that the sound is not only in tune, but also is played as soon as an input is received; any delay can cause the instrument to be difficult to play. We had initially started with only using one Raspberry Pi core (via Python Threading), which resulted in a severely limited max frequency. So, through switching to use all four Raspberry Pi cores (via Python multiprocessing), we were able to meet timing requirements and play the tuned musical notes.

Overall, we were able to get the motors to sound pretty nice, almost like a chiptune (albeit a bit quiet, hence the plastic cups as amplifiers). Our system also played similar to a piano, so once we had the keyboard mapping down, we had a lot of fun messing around with various simple piano tunes. Since our system supports up to three notes, we could even play some cool chords.

Future Work

When we first pitched our proposal, we had two goals in mind: to implement our system as a piano, and to play Midi files on our system. We were aware that we may not have enough time to complete both goals, so the second goal is something we would like to work on in the future. This would entail learning how to parse Midi files and translating their instructions into something our system is capable of playing. If we had more time, we would also have worked on the PyGame screen to add instructions on how to use our project and set up crontab so that we could schedule our program to run on startup.

Parts List

• Nema Stepper Motors (3 pack): $25.99
• A4988 Stepper Motor Drivers (3 pack): $7.99
• Raspberry Pi, PiTFT screen, breadboard, protoboard, power supply, keyboard, mouse, monitor: $0 (borrowed from lab)
• Plastic cups: $0 (from Collegetown Bagels)

Total: $33.91 (+ about $6 for shipping)

References

[1] “A4988 Stepper Motor Driver Module with Heat Sink for 3D Printer Reprap (Pack of 5 Pcs).” StepperOnline, www.omc-stepperonline.com/a4988-stepper-motor-driver-module-with-heat-sink-for-3d-printer-reprap-pack-of-5-pcs-5-a4988.
[2] “Multiprocessing - Process-Based Parallelism.” Python Documentation, docs.python.org/3/library/multiprocessing.html.
[3] “NumPy Documentation.” NumPy Documentation - NumPy v1.26 Manual, numpy.org/doc/stable/index.html.
[4] Pygame.Key - Pygame v2.6.0 Documentation, www.pygame.org/docs/ref/key.html#pygame.key.
[5] Sammy. “Music Note Frequency Chart - Music Frequency Chart.” MixButton, 12 May 2024, mixbutton.com/mixing-articles/music-note-to-frequency-chart/.
[6] Whiteshadow11, and Instructables. “Make Music with Stepper Motors!” Instructables, Instructables, 27 Jan. 2020, www.instructables.com/Make-Music-With-Stepper-Motors/.

import time
import RPi.GPIO as gpio
from threading import Thread
import pygame
from pygame.locals import *
import sys
import os
import numpy as np
from multiprocessing import Process, Queue
import multiprocessing

def GPIO27_callback(channel): #GPIO 27 quit button
  global code_run #python global
  with code_run.get_lock():
    code_run.value = False
def motor1(m1state, direct, tempo, octave, dirPin, stepPin, code_run): #Plays note num for duration dur (in ms)
  # ~ #direct = not direct
  # ~ if(direct):
    # ~ gpio.output(dirPin, gpio.HIGH)
  # ~ else:
    # ~ gpio.output(dirPin, gpio.LOW)
  while(code_run.value):
    if(m1state.value != 0):
      dell = (m1state.value*octave)/10
      gpio.output(stepPin, gpio.HIGH)
      time.sleep(dell/1000000)
      gpio.output(stepPin, gpio.LOW)
      time.sleep(dell/1000000)
      
def motor2(m1state, direct, tempo, octave, dirPin, stepPin, code_run): #Plays note num for duration dur (in ms)
  # ~ #direct = not direct
  # ~ if(direct):
    # ~ gpio.output(dirPin, gpio.HIGH)
  # ~ else:
    # ~ gpio.output(dirPin, gpio.LOW)
  while(code_run.value):
    if(m1state.value != 0):
      dell = (m1state.value*octave)/10
      gpio.output(stepPin, gpio.HIGH)
      time.sleep(dell/1000000)
      gpio.output(stepPin, gpio.LOW)
      time.sleep(dell/1000000)

def motor3(m1state, direct, tempo, octave, dirPin, stepPin, code_run): #Plays note num for duration dur (in ms)
  # ~ #direct = not direct
  # ~ if(direct):
    # ~ gpio.output(dirPin, gpio.HIGH)
  # ~ else:
    # ~ gpio.output(dirPin, gpio.LOW)
  while(code_run.value):
    if(m1state.value != 0):
      dell = (m1state.value*octave)/10
      gpio.output(stepPin, gpio.HIGH)
      time.sleep(dell/1000000)
      gpio.output(stepPin, gpio.LOW)
      time.sleep(dell/1000000)

stepPin1 = 6 #step GPIO pin
dirPin1 = 26 #direction GPIO pin
stepPin2 = 20 #step GPIO pin
dirPin2 = 21 #direction GPIO pin
stepPin3 = 19 #step GPIO pin
dirPin3 = 16 #direction GPIO pin

gpio.setmode(gpio.BCM)
gpio.setup(27, gpio.IN, pull_up_down=gpio.PUD_UP)
gpio.add_event_detect(27, gpio.FALLING, callback=GPIO27_callback, bouncetime=300)

gpio.setup(dirPin1, gpio.OUT) #Dir
gpio.setup(stepPin1, gpio.OUT) #Step
gpio.setup(dirPin2, gpio.OUT) #Dir
gpio.setup(stepPin2, gpio.OUT) #Step
gpio.setup(dirPin3, gpio.OUT) #Dir
gpio.setup(stepPin3, gpio.OUT) #Step

direct = False
use=180
tempo=120
octave=5
freq = 1000
factor = 1

c4=1748
cs4=1640
d4= 1535
ds4=1442
e4=1352
f4=1265
fs4=1185
g4= 1118
gs4=1045
a4=981
as4=917
b4=855
c5= 800
cs5 = 760
d5 = 710

m1state = multiprocessing.Value('i',0)
m2state = multiprocessing.Value('i',0)
m3state = multiprocessing.Value('i',0)

pygame.init()
lcd = pygame.display.set_mode((320, 240))
lcd.fill((0,0,0))
pygame.display.update()
keys = {97:c4, 119: cs4, 115:d4, 101:ds4, 100:e4, 102:f4, 116:fs4, 103:g4, 121: gs4, 104:a4, 117:as4, 106:b4, 107:c5, 111: cs5, 108: d5}
keystate = {97:0, 119: 0, 115:0, 101:0, 100:0, 102:0, 116:0, 103:0, 121: 0, 104:0, 117:0, 106:0, 107:0, 111: 0, 108: 0}
prev_keystate = {97:0, 119: 0, 115:0, 101:0, 100:0, 102:0, 116:0, 103:0, 121: 0, 104:0, 117:0, 106:0, 107:0, 111: 0, 108: 0}

WHITE = (255,255,255)
font_big = pygame.font.Font(None, 25)
text_surface = font_big.render('Stepper', True, WHITE)
rect = text_surface.get_rect(center=(160,100))
lcd.blit(text_surface, rect)
text_surface = font_big.render('Symphony!', True, WHITE)
rect = text_surface.get_rect(center=(159,120))
lcd.blit(text_surface, rect)
pygame.display.update()

code_run = multiprocessing.Value('b', True)
p1 = Process(target = motor1, args = (m1state, direct, tempo, octave, dirPin1, stepPin1, code_run,))
p2 = Process(target = motor2, args = (m2state, direct, tempo, octave, dirPin2, stepPin2, code_run,))
p3 = Process(target = motor3, args = (m3state, direct, tempo, octave, dirPin3, stepPin3, code_run,))
p1.start()
p2.start()
p3.start()

while (code_run.value):
  if(code_run.value == False):
    break
  # Scan keyboard events
  for event in pygame.event.get():
    if(event.type == pygame.KEYDOWN or event.type == pygame.KEYUP):
      #Get pressed keys
      keys_pressed = np.nonzero(pygame.key.get_pressed())
      keys_pressed = keys_pressed[0]
      
      if (event.key == K_z):
        factor = 1/2.185
      elif (event.key == K_x):
        factor = 1
      
      #Get notes of pressed keys
      for i in keys:
        if(np.isin(i,keys_pressed)):
          prev_keystate[i] = keystate[i]
          keystate[i] = 1
        else:
          prev_keystate[i] = keystate[i]
          keystate[i] = 0
        if(prev_keystate[i]==0 and keystate[i] ==1): #rising edge
          #start motor at note if motor is off
          if m1state.value == 0: 
            with m1state.get_lock():
              m1state.value = int(keys[i]/factor)
          elif(m2state.value == 0):
            with m2state.get_lock():
              m2state.value = int(keys[i]/factor)
          elif(m3state.value == 0):
            with m3state.get_lock():
              m3state.value = int(keys[i]/factor)
            
        if(prev_keystate[i]==1 and keystate[i] ==0): #falling edge
          #stop motor at note if playing
          if m1state.value == int(keys[i]/factor):
            with m1state.get_lock():
              m1state.value = 0
          if m2state.value == int(keys[i]/factor):
            with m2state.get_lock():
              m2state.value = 0
          if m3state.value == int(keys[i]/factor):
            with m3state.get_lock():
              m3state.value = 0
      
p1.join()
p2.join()
p3.join()
gpio.output(stepPin1, gpio.LOW)
gpio.output(dirPin1, gpio.LOW)
gpio.output(stepPin2, gpio.LOW)
gpio.output(dirPin2, gpio.LOW)
gpio.output(stepPin2, gpio.LOW)
gpio.output(dirPin2, gpio.LOW)
gpio.cleanup()
              

import time
import RPi.GPIO as gpio
from threading import Thread
import pygame
from pygame.locals import *
import sys
import os
import numpy as np

def GPIO27_callback(channel): #GPIO 27 quit button
  global code_run #python global
  code_run = False #this says that if code_run is somewhere, it is a global var (NOT DECLARATION)

def motor1(): #Plays note num for duration dur (in ms)
  global m1state #motor 1 state
  #Timing
  global direct
  global tempo
  global octave
  global dirPin
  global stepPin
  # ~ #direct = not direct
  # ~ if(direct):
    # ~ gpio.output(dirPin, gpio.HIGH)
  # ~ else:
    # ~ gpio.output(dirPin, gpio.LOW)
  
  #coun = int((dur*5*tempo)/dell)
  while(code_run):
    if(m1state != 0):
      dell = (m1state*octave)/10
      gpio.output(stepPin, gpio.HIGH)
      time.sleep(dell/1000000)
      gpio.output(stepPin, gpio.LOW)
      time.sleep(dell/1000000)

stepPin = 6 #step GPIO pin
dirPin = 26 #direction GPIO pin
gpio.setmode(gpio.BCM)
gpio.setup(27, gpio.IN, pull_up_down=gpio.PUD_UP)
gpio.add_event_detect(27, gpio.FALLING, callback=GPIO27_callback, bouncetime=300)
gpio.setup(dirPin, gpio.OUT) #Dir
gpio.setup(stepPin, gpio.OUT) #Step

direct = False
use=180
tempo=120
octave=5
freq = 1000

c4=1748
cs4=1640
d4= 1535
ds4=1442
e4=1352
f4=1265
fs4=1185
g4= 1118
gs4=1045
a4=981
as4=917
b4=855
c5 = 800

m1state = c4

pygame.init()
lcd = pygame.display.set_mode((320, 240))
lcd.fill((0,0,0))
pygame.display.update()
keys = {97:c4,115:d4,100:e4,102:f4,103:g4,104:a4,106:b4,107:c5}
keystate = {97:0,115:0,100:0,102:0,103:0,104:0,106:0,107:0}
prev_keystate = {97:0,115:0,100:0,102:0,103:0,104:0,106:0,107:0}
t1 = Thread(target = motor1)
t1.start()
code_run = True

while (code_run):
  if(code_run == False):
    break
  # Scan keyboard events
  for event in pygame.event.get():
    if(event.type == pygame.KEYDOWN or event.type == pygame.KEYUP):
      #Get pressed keys
      keys_pressed = np.nonzero(pygame.key.get_pressed())
      keys_pressed = keys_pressed[0]
      
      #Get notes of pressed keys
      for i in keys:
        if(np.isin(i,keys_pressed)):
          prev_keystate[i] = keystate[i]
          keystate[i] = 1
        else:
          prev_keystate[i] = keystate[i]
          keystate[i] = 0
        if(prev_keystate[i]==0 and keystate[i] ==1): #rising edge
          if m1state == 0: 
            #start motor at note if motor is off
            m1state = keys[i]
        if(prev_keystate[i]==1 and keystate[i] ==0): #falling edge
          if m1state == keys[i]:
            #stop motor at note if playing
            m1state = 0

      
t1.join()
gpio.output(stepPin, gpio.LOW)
gpio.output(dirPin, gpio.LOW)
gpio.cleanup()
              

import time
import RPi.GPIO as gpio
from threading import Thread
import pygame
from pygame.locals import *
import sys
import os
import numpy as np


def GPIO27_callback(channel): #GPIO 27 quit button
  global code_run #python global
  code_run = False #this says that if code_run is somewhere, it is a global var (NOT DECLARATION)

stepPin = 6 #step GPIO pin
dirPin = 26 #direction GPIO pin
gpio.setmode(gpio.BCM)
gpio.setup(27, gpio.IN, pull_up_down=gpio.PUD_UP)
gpio.add_event_detect(27, gpio.FALLING, callback=GPIO27_callback, bouncetime=300)
gpio.setup(dirPin, gpio.OUT) #Dir
gpio.setup(stepPin, gpio.OUT) #Step


pygame.init()
lcd = pygame.display.set_mode((320, 240))
lcd.fill((0,0,0))
pygame.display.update()
keys = {97:'a',115:'s',100:'d',102:'f',103:'g',104:'h',106:'j',107:'k',108:'l'}

code_run = True

while (code_run):
  if(code_run == False):
    break
  # Scan keyboard events
  for event in pygame.event.get():
    #If any keyboard input is detected:
    if(event.type == pygame.KEYDOWN or event.type == pygame.KEYUP):
      #Get pressed keys
      keys_pressed = np.nonzero(pygame.key.get_pressed())
      keys_pressed = keys_pressed[0]
      print(keys_pressed)
      
gpio.output(stepPin, gpio.LOW)
gpio.output(dirPin, gpio.LOW)
gpio.cleanup()
              

import time
import RPi.GPIO as gpio
from threading import Thread

def GPIO27_callback(channel): #GPIO 27 quit button
  global code_run #python global
  code_run = False #this says that if code_run is somewhere, it is a global var (NOT DECLARATION)
  
def note(num,dur): #Plays note num for duration dur (in ms)
  global direct
  global tempo
  global octave
  global dirPin
  global stepPin
  
  dell = (num*octave)/10
  #direct = not direct
  if(direct):
    gpio.output(dirPin, gpio.HIGH)
  else:
    gpio.output(dirPin, gpio.LOW)
  
  coun = int((dur*5*tempo)/dell)
  for x in range(coun):
    gpio.output(stepPin, gpio.HIGH)
    time.sleep(dell/1000000)
    gpio.output(stepPin, gpio.LOW)
    time.sleep(dell/1000000)

def pa(durp): #pause between notes for duration durp (ms)
  global tempo
  ker=int(durp/100)*tempo
  print("pause")
  print(ker)
  time.sleep(ker/1000)

stepPin = 6 #step GPIO pin
dirPin = 26 #direction GPIO pin
gpio.setmode(gpio.BCM)
gpio.setup(27, gpio.IN, pull_up_down=gpio.PUD_UP)
gpio.add_event_detect(27, gpio.FALLING, callback=GPIO27_callback, bouncetime=300)
gpio.setup(dirPin, gpio.OUT) #Dir
gpio.setup(stepPin, gpio.OUT) #Step

direct = False
use=180
tempo=120
octave=5
freq = 1000

c4=1748
cs4=1640
d4= 1535
ds4=1442
e4=1352
f4=1265
fs4=1185
g4= 1118
gs4=1045
a4=981
as4=917
b4=855
c5 = 800
code_run = True

while (code_run):
  if(code_run == False):
    break
  note(c4,1000)
  pa(500)
  note(d4,1000)
  pa(500)
  note(e4,1000)
  pa(500)
  note(f4,1000)
  pa(500)
  note(g4,1000)
  pa(500)
  note(a4,1000)
  pa(500)
  note(b4,1000)
  pa(500)
  note(c5,1000)
  pa(500)

gpio.output(stepPin, gpio.LOW)
gpio.output(dirPin, gpio.LOW)
gpio.cleanup()